Constrained estimation and the theorem of Kuhn-Tucker

There are many statistical problems in which the parameter of interest is restricted to a subset of the parameter space. The constraint(s) may reflect prior knowledge about the value of the parameter, or, may be a device used to improve the statistical properties of the estimator. Estimation and inferential procedures for such models may be derived using the theorem of Kuhn-Tucker (KT). The theorem of KT is a theorem in nonlinear programming which extends the method of Lagrange multipliers to inequality constraints. KT theory characterizes the solution(s) to general constrained optimization problems. Often, this characterization yields an algorithmic solution. In general, though, this is not the case and the theorem of KT is used together with other tools or algorithms. For example, if the constraints are linear or convex, then the tools of convex optimization (Boyd and Vandenberghe [2]) may be used; of these linear and quadratic programming are best known. More generally, interior point methods, a class of iterative methods in which all iterations are guaranteed to stay within the feasible set, may be used. Within this class, Lange [12] describes the adaptive barrier method with statistical applications. Geyer and Thompson [6] develop a Monte-Carlo method for constrained estimation based on a simulation of the likelihood function. Robert and Hwang [16] develop the prior feedback method. They show that the constrained estimator may be viewed as the limit of a sequence of formal Bayes estimators. The method is implemented using MCMCmethodology. In some situations constrained problems may be reduced to isotonic regression problems. A variety of algorithms for solving isotonic regression are